Graph Structure Learning for Task Ordering

In many practical applications, multiple interrelated tasks must be accomplished in sequential order through user interactions with multiple retrieval, classification and recommendation systems. The ordering of the tasks may have a significant impact on the overall utility (or performance); hence optimal ordering of tasks is desirable. However, manual specification of near-optimal ordering is often difficult as the number of tasks increases and as the complexity of dependencies among tasks grows. This paper proposes an automated solution for this difficult problem. We use an empirically learned graph to represent partial-order preferences among tasks: the nodes are tasks, and the edges are directed and weighted. The weight of each edge is the expected utility enhancement when the preferred partial order of the task pair is reinforced. We learn the graph structure empirically by running SVM with multiple classification tasks in randomly generated orders, and by averaging the utility scores (e.g., measured using F1) conditioned on the partial order of each task pair. The induced graph allows us to use link analysis (e.g., with HITS) to rank tasks based on how important they are in the graph and in terms of satisfying and propagating partial-order preferences. In our experiments on a large collection of business proposals from the Accenture Consulting & Technology Company, the performance of the proposed method was significantly better than that of random ordering and virtually equal to the utility of expert-suggested ordering.